Emergence of cooperation induced by preferential learning 



O 

o 

(N 



o 

Q 
• 1— I 

Oh 



> 

o 
o 
m 
o 
\o 
o 

o 

Oh 



Jie Ren^, Wen-Xu Wang^, Gang Yan'^, and Bing-Hong Wang^ 
^Department of Physics, 
^Department of Modern Physics, 
Department of Electronic Science and Technology, 
Universtty of Science and Technology of China, 
Hefei, 230026, PR China 
(Dated: February 2, 2008) 

The evolutionary Prisoner's Dilemma Game (PDG) and the Snowdrift Game (SG) with preferen- 
tial learning mechanism are studied in the Barabasi- Albert network. Simulation results demonstrate 
that the preferential learning of individuals remarkably promotes the cooperative behavior for both 
two games over a wide range of payoffs. To understand the effect of preferential learning on the 
evolution of the systems, we investigate the time series of the cooperator density for different pref- 
erential strength and payoffs. It is found that in some specific cases two games both show the 
l//-scaling behaviors, which indicate the existence of long range correlation. We also figure out 
that when the large degree nodes have high probability to be selected, the PDG displays a punctu- 
ated equilibrium-type behavior. On the contrary, the SG exhibits a sudden increase feature. These 
temporary instable behaviors are ascribed to the strategy shift of the large degree nodes. 

PACS numbers: 87.23.Kg, 02.50.Le, 87.23. Ge, 89.75.CC 



I. INTRODUCTION 



Cooperation is ubiquitous in real world, ranging from 
biological systems to economic and social systems jj. 
However, the unselfish, altruistic actions apparently con- 
tradict Darwinian selection. Thus, understanding the 
conditions for the emergence and maintenance of co- 
operative behavior among selfish individuals is a cen- 
tral problem Q. Game theory together with its ex- 
tensions 0, 0, iM IE 111 I considered to be an impor- 
tant approach, provides a useful framework for inves- 
tigating this problem. Two simple games. Prisoners' 
Dilemma Game (PDG) 1^ and Snowdrift Game (SG) 
(iQj . as metaphors for characterizing the evolution of 
cooperative behavior have drawn much attention from 
not only social but also biological and physical scientists 

[n m m El El E la Ls; 19, 20, 21, 22, ,23, 24, 2j . 

In the original PDG, each of two players may chose ei- 
ther to cooperate or defect in any one encounter. If they 
both cooperate, both obtain a payoff of R, whereas mu- 
tual defection results in pay-off P for both players. If one 
player defects while the other cooperates, defector gains 
the biggest pay-off T, while cooperator gets S. The rank- 
ing of four pay-off values is T>R>P>S. The SG 
is a game of much biologically interesting. This game 
differs from the PDG mainly in the order of P and S, as 
T > R > S > P, which are more favorable to sustain 
cooperative behavior. 

However, in these two games, the unstable coopera- 
tive behavior is opposite to the observations in the real 
world. This disagreement thus motivates a variety of 
suitable extensions of basic model rules to explain the 
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emergence of cooperation. Some previous works have 
suggested that the introduction of "tit-for-tat" H 0| 
strategy can remarkably enhance the cooperative behav- 
ior. More recently, Nowak and May E3l found that the 
PDG with simple spatial structure can induce the emer- 
gence of cooperation, and in particular, spatial chaos 
is observed. In contrast, the work of Hauert and Doe- 
beli [23 demonstrates the spatial structure often inhibits 
the evolution of cooperation in the SG. Inspired by the 
idea of spatial game, much attention has been given to 
the interplay between evolutionary cooperative behav- 
ior and the underlying structure [ll IitI 111 ITl Jg^ l25l| . 
Since the surprising discovery of "small world" |2y| and 
"scale-free" [23 structural properties in real networked 
systems, evolutionary games are naturally considered on 
the networks with these two kinds of structural features 
miMIIllllirS]. Interestingly, it is found that compar- 
ing with the square lattices. Scale- free networks provide 
a unifying framework for the emergency of cooperation 

In the two games with network structure, such as 
square lattices (spatial structure), small world and scale- 
free structure, players interact only with their immediate 
neighbors. In each round, the score of each individual is 
the sum of the payoffs in the encounters with its neigh- 
bors. At the next generation, all the individuals could up- 
date their strategies (cooperate or defect) synchronously 
according to either the deterministic rule introduced by 
Nowak and May or the stochastic evolutionary rule 
by Szabo and Toke jl7l|. 

In this paper, we focus on the PDG and SG on scale- 
free networks mainly according to the stochastic update 
rules. However, we argue that such as in the social sys- 
tem, individual may not completely randomly choose a 
neighbor to learn from it. "Rich gets richer" is a common 
feature in social and natural system, which reveals the ex- 
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istence of preferential mechanism. It is indeed the pref- 
erential attachment mechanism of Barabasi and Albert 
model (BA for short) [23| leads to the scale-free struc- 
tural property in good accord with the empirical obser- 
vations. Thus, in the present work, we present a prefer- 
ential learning rule, the probability of which is governed 
by a single parameter, for better mimicking the evolu- 
tion of real world system. The probability of choosing 
a neighbor for each individual depends on the degree of 
that neighbor. This assumption takes into account that 
the status of individuals can be reflected by the degree 
of them in various communities in nature and society, 
e.g. the leader usually interacts with large quantities of 
individuals. Interestingly, we find that the preferential 
learning mechanism promotes the cooperative behavior 
of both the PDG and SG. Several attractive properties for 
some specific parameter values are observed, such as the 
1 //-like noise of evolutionary cooperator density for both 
two games, which indicates the long range correlation of 
cooperation. In the SG, for some specific cases, the de- 
gree of cooperation displays a punctuated equilibrium- 
type behavior instead of steady state. In contrast, the 
PDG exhibits an absolutely different property of sudden 
jumps of cooperation. These two distinct behaviors are 
both attributed to the effect of leaders, i.e. the individ- 
uals with large connectivity. 

The paper is arranged as follows. In the following sec- 
tion we describe the model in detail, in Sec. Ill simu- 
lations and analysis are provided for both the PDG and 
SG , and in Sec. IV the work is concluded. 



II. THE MODEL 

We first construct the scale- free networks using the BA 
model which is considered to be the most simple and 
general one. Starting from ttiq fully connected nodes, 
one node with m links is attached at each time step in 
such a way that the probability 11^ of being connected 
to the existing node i is proportional to the degree ki 
of that node, i.e., 11^ = ki/^^ kj with summation over 
all the existing nodes. Here, we set m = mo = 2 and 
network size N — 5000 for all simulations. The degree 
distribution of BA networks follows a power law P{k) ^ 

We consider the evolutionary PDG and SG on the net- 
works. Without losing generality, we investigate the sim- 
plified games with a single payoff parameter following 
previous works [isl IT^ E^ . Figure (1) illustrates the 
encounter payoffs of both the PDG and SG. Each indi- 
vidual is placed on a node of the network and plays the 
games only with their immediate neighbors simultane- 
ously. The total payoff of each player is the sum over all 
its encounters. 

During the evolutionary process, each player is allowed 
to learn from one of its neighbors and update its strategy 
in each round. As mentioned early, each player chooses a 
neighbor according to the preferential learning rule, i.e.. 
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FIG. 1: The payoffs: In the PDG, when two cooperators (C) 
encounter, both earn 1. While two defectors (D) encounter, 
both earn 0. When a cooperator encounters a defector, the 
defector earns b and the cooperator 0. In the SG, it is the 
same as PDG when two cooperators or defectors encounter. 
However, when a cooperator meets a defector the cooperator 
scores 1 — r and the defector scores 1 +r. 



the probability Pi^j of i selecting a neighbor j is 



(1) 



where a is a tunable parameter and the sum runs over 
the neighbors of i. One can see when a equals zero, the 
neighbor is randomly selected so that the game is re- 
duced to the original one. While in the case of a > 0, 
the individuals with large degree have advantages to be 
selected; Otherwise, the small degree individuals have 
larger probability to be selected. In social and natural 
systems, some individuals with high status and reputa- 
tion may have much stronger influence than others and 
the status of individuals can be reflected by the degree of 
them. Thus, the introduction of the preferential learning 
intends to characterize the effect of influential individuals 
on the evolution of cooperation. In parallel, we also in- 
vestigate the performance of the systems with tendency 
of learning from the individuals with small degree. 

After choosing a neighbor y, the player x will adopt 
the coplayer's strategy with a probability depending on 
the normalized total payoff difference presented in Rcf. 
in as 



W 



1 



1 + eXp[(A'4/fc^ - My/ky)/T] ' 



(2) 



where and My are the total incomes of player x and 
y, and T characterizes the noise effects, including fluctu- 
ations in payoffs, errors in decision, individual trials, etc. 
This choice of W takes into account the fact of bounded 
rationality of individuals in sociology and reffects nat- 
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FIG. 2: (color online). The cooperator density rhoc of the 
PDG as a function of parameter a for different value of b. 

ural selection based on relative fitness in terms of evo- 
lutionism. The ratio of total income of individual and 
its degree, i.e., M^/k^ denotes the normalized total pay- 
off. This normalization avoids an additional bias from 
the different degree of nodes. In the next section, we 
perform the simulations of the PDG and SG respectively 
and our goal is to find how the preferential learning affect 
the evolutionary cooperative behaviors of both PDG and 
SG. 



III. SIMULATION RESULTS 

The key quantity for characterizing the cooperative be- 
havior of the system is the density of cooperators pc- 
Hence, we first investigate pc as a function of the tunable 
parameter a for different payoff parameter b in the PDG, 
as shown in Fig. 2. The simulation results were obtained 
by averaging over last 10000 time steps of entire 20, 000 
time steps. Each data point results from an average over 
20 simulations for the same type of network structure. 
In the initial state, the strategies of C and D are uni- 
formly distributed among all the players. We figure out 
that, comparing with the case of no preferential learning, 
i.e.,Q = 0, the cooperation is remarkably promoted not 
only for positive value of a, but also for negative a in 
a wide range of b. For negative a, the pc monotonously 
increases with the decrease of a and finally pc reaches a 
upper limit for very small a. In contrast, in the case of 
positive a, we find that pc increases dramatically and 
there exists a maximal value of pc, which indicates that 
although the leaders with large degree play a key role in 
the cooperation, very strong influence of leaders will do 
harm to the persistence of cooperation and make indi- 
viduals to be selfish. One can also find that the larger 
the value of 6, the larger the value of a corresponding to 
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FIG. 3: (color online). The cooperator density pc of the SG 
as a function of parameter a for different value of r. 



the maximal pc- Moreover, an interesting phenomenon 
is observed in Fig. 2, that is when b is small, positive 
a leads to better cooperative behavior than the negative 
one; However, for large b, the system performs better 
when choosing negative a. These results imply that if the 
income of defectors is only little more than that of coop- 
erators, the leader's effect will considerably enhances the 
cooperation; While if the selfish behavior is encouraged 
in the system (large b), the influential individuals will 
leads to the imitation of selfish behavior and reduce the 
cooperator density in a certain extent. On the contrary, 
restriction of leader's influence (negative a decreases the 
selected probability of large degree individuals by their 
neighbors) results in better cooperation. 

In parallel, we investigate the effect of preferential 
learning upon the SG. The simulation results are demon- 
strated in Fig. 3. Similar to the PDG, Pc is improved 
by the introduction of preferential learning for nearly the 
entire range of r from to 1. In both sides of a = 0, 
Pc reaches an upper limit, which means that in the case 
of strong leader's influence or without leaders, coopera- 
tion can be promoted to the highest level for the wide 
middle range of b. Contrary to the PDG, for very large 
r, the system still performs cooperative behavior, which 
is attributed to the fact that the rule of SG favors the 
cooperators, that is the cooperators yet gain payoff 1 — r 
when meeting defectors. Combining the above simula- 
tion results of both the PDG and SG, we can conclude 
that the preferential learning mechanism indeed plays an 
important role in the emergence of cooperation. 

In the following, we analyze the time series of the coop- 
erator density to give detailed description of the systems' 
evolutionary behavior. We first study the PDG for nega- 
tive value of parameter a. Surprisingly, for some specific 
values of b and a, l//-like noise is found. A prototyp- 
ical example is exhibited in Fig. 4. The l//-like noise 
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FIG. 4: (a) Time series of cooperator density rhoc of the 
PDG for b = 1.0 and a = —1. (b) Power spectrum analysis 
of (a). 



FIG. 5: (a) Time series of cooperator density rhoc of the SG 
for r = 0.5 and a — —5. (b) Power spectrum analysis of (a). 



is observed frequently in real- worl d sy stems, including 
healt hy p hysiologic systems [23, 123, 123 ) economical sys- 
tems jsJIs^, as well as traffic systems Issj. However, 
as far as we know, 1// pattern hasn't been reported in 
the study of evolutionary games. The 1// noise denotes 
that the power spectrum of time series varies as a power- 
law S{f) ~ /""^ with the slope 0=1. The spectrum 
exponent </> characterizes the nature of persistence or the 
correlation of the time series, (j) = 2 indicates zero corre- 
lation associated with Brownian motion, where as = 
corresponds to a completely uncorrelated white noise. 
4> > 2 indicates positive correlation and persistence i.e., 
if the process was moving upward (downward) at time t, 
it will tend to continue to move upward (downward) at 
future times t'; (/) < 2 represents negative correlation and 
anti-persistence. The intermediate case, S{f) ^ f~'^, is a 
"compromise" between the small-time-scale smoothness 
and large-time-scale roughness of Brownian noise. Figure 
4 (a) shows the time evolution for b = 1.0 and a — —1, 
i.e. the case of restriction of leader's influence. In this 
case, the density of cooperators remains stable with fre- 
quently fluctuations around the average value. Figure 4 
(b) is the power spectrum analysis of the time series of 
cooperator density. A prototypical 1 / /-hke noise is found 
with the fitted slope cj) = 1.06. This result indicates when 
the small degree individuals have large probability to be 
followed, i.e., suppress the influential leader's effect, the 
nontrivial long range correlation of evolutionary cooper- 
ative behavior emerges. The similar phenomenon is also 
observed in the SG for the case of negative a, as shown 
in Fig. 5. The emergence of the 1// scaling is associ- 
ated with the parameter values a = —1 and r = 0.5. 
The discovered 1/f noise for both two games is partly 
ascribed to the lack of influence of leaders. Suppose that 



if the individuals with large connectivity are chosen with 
large probability, their strategy will be easily followed by 
a large number of persons, because those leaders usually 
gain very high income. Since the influential ones only 
take the minority, the evolutionary cooperative behav- 
ior will mainly determined by the minority. Besides, the 
strategies of those leaders are usually fixed due to their 
very high score, the long range correlation of the fluctu- 
ation of cooperator density is broken. 

Then we investigate the evolutionary behavior of both 
the SG and PDG in the case of positive a. For the SG, 
when the parameter a is close to zero, for arbitrary b, the 
level of cooperation remains stable with relatively small 
fluctuations around the average value. This property is 
remarkably changed for large value of a, which means the 
influence of leaders becomes strong. As shown in Fig. 6, 
for a = 5 and r = 0.5, the equilibrium is punctuated 
by sudden drops of cooperator density. After a sudden 
drop, the cooperation level pc will gradually increase un- 
til PC reaches the average value. The occurrence of these 
punctuated equilibrium-type behavior is ascribed to the 
strong influence of a small amount of leaders. As we 
have mentioned, the leader nodes usually get large pay- 
offs, thus they tend to hold their own strategies and are 
not easily affected by their neighbors. However, those 
influential individuals still have small probability to fol- 
low their neighbors' strategies. If an event that a leader 
shift his strategy to defector occasionally happens, the 
successful defector strategy will rapidly spread from the 
leader to his vicinities. Due to the connection heterogene- 
ity of the scale-free networks, i.e., the leaders have large 
amount of neighbors, the imitation of a successful self- 
ish behavior of the leader triggers the rapidly decrease 
of cooperator density. After the occurrence of a sud- 
den drop, defectors become the majority and the selfish 
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the other leaders with cooperate strategies, the coopera- 
tor density will slowly recover to the steady state. 

The evolutionary behavior of the PDG for the posi- 
tive a also exhibits nontrivial feature as shown in Fig. 
7. Contrast to the SG, the cooperation level shows some 
sudden increases. The mechanism that induces the tem- 
porary instability of cooperator density is the same as 
that of the sudden drops of the SG. The strategy shift 
of influential nodes plays the main role in the occurrence 
of the sudden increase. Opposite to the SG, the payoff 
matrix of the PDG favors the defect behavior, thus the 
cooperation level is quite low. An occasional strategy 
shift from defect to cooperate of a leader will lead to the 
imitation of its neighbors and a sudden increase occurs. 
However, the high cooperator density is instable in the 
PDG for large b, hence the sudden increase will rapidly 
decrease to the average value. 



FIG. 6: The time evolution of cooperator density of the SG 
with r = 0.5, a = 5 exhibits the punctuated equilibrium-type 
behavior. 
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FIG. 7: The sudden increase of cooperator density of the 
PDG with b = 1.5, a = l. 



leader nearly gain nothing. Then under the influence of 



IV. CONCLUSION AND DISCUSSION 

We have investigated the cooperative behavior of the 
evolutionary games resulting from the preferential mech- 
anism. Comparing with the cases of random selection, 
i.e., a = 0, preferentially selecting large degree nodes or 
small degree ones can promote the cooperator density of 
both the PDG and the SG over a wide range of payoffs. 
For the cases of negative value of a, the systems perform 
the behavior of long range correlation, which is quanti- 
fied by the 1/f scaling of power spectrum. Interestingly, 
in the case of positive value of a, i.e., the large degree 
nodes have high probability to be selected for imitation, 
the SG exhibits a punctuated equilibrium-type behavior 
which is qualified by the occasional occurrence of sudden 
drops. In contrast to the SG, the PDG shows temporary 
instable behavior with the existence of sudden increase. 
The mechanism that leads to the instabilities of cooper- 
ation for both games arc the strategy shift of influential 
nodes and the imitation of their neighbors. The insta- 
ble behavior indicates that the strong influence of leader 
individuals will do harm to the evolutionary cooperative 
behavior of the systems. The present work implies that 
the existence of the preferential learning mechanism plays 
an important role in the emergence of cooperation in the 
heterogeneous networked systems. 
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